Serial modulation display having binary light modulation stage

ABSTRACT

A display has first and second spatial light modulators for modulating light from a light source. The first spatial light modulator has a plurality of elements switchable between ON and OFF states according to a pattern having a spatially-varying density. Transfer optics blur and carry light modulated by the first spatial light modulator to the second spatial light modulator to yield a light field at the second spatial light modulator. The second spatial light modulator has a plurality of elements switchable between ON and OFF states to perform temporal dithering of the light field to provide a reconstruction of the image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.12/021,206 entitled SERIAL MODULATION DISPLAY HAVING BINARY LIGHTMODULATION STAGE filed on 28 Jan. 2008.

TECHNICAL FIELD

The invention relates to electronic displays such as computer displays,television displays, digital cinema projectors, home theatre displays,displays in simulators for vehicles such as aircraft, ships, trucks,cars and the like, gaming system displays, displays in simulation-typeamusement rides, digital picture frames, HDTV monitors, high dynamicrange (HDR) imaging systems and the like. The invention relatesparticularly to displays wherein light is modulated in two stages.

BACKGROUND

Electronic displays are used in a wide range of applications. Someelectronic displays have a spatial light modulator. Elements of thespatial light modulator are controlled in response to image data toyield an image that can be observed by viewers. The elements of somespatial light modulators are ‘binary’ elements which have two states. Inone state the element passes light to a viewing area and in anotherstate the element does not pass light to the viewing area.

A digital mirror device (DMD) is one example of a binary spatial lightmodulator. A DMD provides an array of mirrors. Each mirror can beswitched between two states. The state of a mirror can determine whetheror not light incident on the DMD at the location of the mirror will passalong a path that will take it to a viewing area. When a mirror is in an‘ON’ state, light is directed to a location in a viewing area thatcorresponds to the mirror. When the mirror is in an ‘OFF’ state thelight is directed along a path that does not take it to the viewingarea. It is typical for light in the OFF state to be directed to a heatsink.

An element of a binary spatial light modulator can be controlled todisplay intermediate brightness values by rapidly turning it on and off.The brightness that will be perceived by a human observer can be alteredby adjusting the relative amounts of time during which the element is inits ON and OFF states.

Some displays provide serial light modulators. In such displays, lightis modulated serially by first and second light modulators. Examples ofdisplays are described in PCT Patent Publication No. WO2003/077013 andU.S. Pat. No. 6,891,672. PCT Patent Publication No. WO2003/077013describes a light source having an array of controllable light-emittingelements, and a spatial light modulator having an array of elements ofcontrollable transmissivity for modulating light from the light source.U.S. Pat. No. 6,891,672 describes first and second spatial lightmodulators arranged in series to modulate light from a light source.Each spatial light modulator has an array of controllable pixels,wherein each pixel of one of the spatial light modulators corresponds toa plurality of pixels of the other one of the spatial light modulators.

There is a need for cost effective displays capable of providing highimage quality.

SUMMARY OF THE INVENTION

This invention has a number of aspects. One aspect of the inventionprovides a display. The display may comprise, for example, a computerdisplay, a television, a digital projector or the like. The displaycomprises a light source capable of directing light onto a first spatiallight modulator. The first spatial light modulator comprises a pluralityof first elements switchable between ON and OFF states. The display hastransfer optics arranged to direct light modulated by the first spatiallight modulator onto a second spatial light modulator and a driverconfigured to generate first and second control signals for the firstand second spatial light modulators respectively based on image data.The driver is configured to generate a pattern based upon the imagedata. The pattern has a spatially-varying density. The pattern maycomprise a spatial dither derived from the image data for example. Thedriver is configured to generate the first control signal so as to setelements of the first spatial light modulator according to the pattern.The transfer optics are characterized by a transfer function that blurslight originating from the first spatial light modulator at the secondspatial light modulator.

The second spatial light modulator may also comprise a plurality ofelements switchable between ON and OFF states. In such a case, thedriver may be configured to switch the elements of the second spatiallight modulator between their ON and OFF states multiple times during animage frame. The switching of the elements of the second spatial lightmodulator may be performed, for example, according to a binarypulse-width modulation scheme.

In some embodiments, the driver is configured to estimate a light fieldat the second spatial light modulator corresponding to the pattern andto base the second control signals on the estimated light field.

Another aspect of the invention provides a display comprising: means forgenerating light; first means for binary modulation of the light, thefirst means comprising a plurality of first elements switchable betweenON and OFF states; means for blurring light modulated by the first meansand directing the blurred light onto a binary spatial light modulator;means for generating first control signals for the first means based onthe image data, the means for generating first control signalscomprising means for generating a pattern based upon the image data, thepattern having a spatially-varying density; and means for generatingsecond control signals for the binary spatial light modulator based onthe image data.

Another aspect of the invention provides a method for displaying animage. The method comprises setting elements of a first binary spatiallight modulator according to a binary pattern based on the image. Thepattern has a spatially-varying density. The method proceeds by blurringand transferring to a second spatial light modulator an image of thefirst binary light modulator to yield a light field at the secondspatial light modulator; and modulating the light field with the secondspatial light modulator to yield a reconstruction of the image.

In some embodiments, modulating the light field with the second spatiallight modulator comprises performing temporal dithering of the lightfield by switching elements of the second spatial light modulatorbetween ON and OFF states.

Some embodiments involve computing an estimate of the light fieldcorresponding to the pattern and controlling the second spatial lightmodulator according to the image data and the estimate of the lightfield. The computed estimate may take into consideration a transferfunction that characterizes the blurring.

Another aspect of the invention provides a controller for a displaycomprising first and second spatial light modulators. The controller isconfigured to generate a first control signal for the first spatiallight modulator to set each of a plurality of elements of the firstspatial light modulator to an ON or OFF state according to a binarypattern having a spatially-varying density based on an image; andgenerate a second control signal for the second spatial light modulatorto switch each of a plurality of elements of the second spatial lightmodulator between ON and OFF states to perform temporal dithering oflight incident on the element. The second control signal is responsiveto an estimated light field of light modulated by the first spatiallight modulator and image data.

Further aspects of the invention as well as features of specificembodiments of the invention are described below.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate non-limiting embodiments of theinvention.

FIG. 1 is a schematic diagram illustrating a monochrome displayaccording to a simple embodiment of the invention.

FIG. 2 is a flow chart which illustrates a method for displaying imagesaccording to an embodiment of the invention.

FIG. 3A illustrates an image having portions with different levels ofbrightness. FIG. 3B illustrates an example of a dithering pattern forrepresenting the image shown in FIG. 3A.

FIG. 4 is a graph illustrating the variation of various characteristicswith position across an image.

FIG. 5 is a block diagram of a controller for a display according to anexample embodiment of the invention connected to control two spatiallight modulators.

FIG. 6 is a schematic diagram illustrating a color display according toanother embodiment of the invention.

FIG. 7 is a schematic diagram illustrating a color display according toanother embodiment of the invention.

FIG. 8 is a block diagram of a controller according to anotherembodiment of the invention.

FIG. 9 is a schematic diagram illustrating a color display according toanother embodiment of the invention.

DESCRIPTION

Throughout the following description, specific details are set forth inorder to provide a more thorough understanding of the invention.However, the invention may be practiced without these particulars. Inother instances, well known elements have not been shown or described indetail to avoid unnecessarily obscuring the invention. Accordingly, thespecification and drawings are to be regarded in an illustrative, ratherthan a restrictive, sense.

FIG. 1 shows a monochrome display 10 according to an example embodimentof the invention. Display 10 comprises a light source 12. Light 13 fromlight source 12 illuminates a first spatial light modulator 14. Lightsource 12 may comprise, for example:

a laser;

a xenon lamp;

an array of laser diodes or other solid-state light emitters;

an arc lamp; or

the like.

First spatial light modulator 14 comprises a plurality of controllableelements 16. Elements 16 can be switched between ON and OFF states by asuitable control circuit 18. When it is in its ON state, an element 16allows incident light 13 that hits the element to pass to acorresponding area of a second spatial light modulator 20. When it is inits OFF state, the amount of light that passes from the element 16 tothe corresponding area of the second spatial light modulator 20 isdiminished. Ideally, when an element 16 is in its OFF state,substantially no light from the element 16 reaches the correspondingarea of the second spatial light modulator 20.

First spatial light modulator 14 may be implemented in a wide variety ofways. First spatial light modulator 14 comprises a DMD in someembodiments. In other embodiments, first spatial light modulator 14comprises an array of optical reflective or transmissive elements thatcan be switched between ON and OFF states by other mechanisms. Forexample, in some such embodiments first spatial light modulator 14comprises an LCD panel. LCOS chip or the like. In other embodiments, thefunctions of light source 12 and first spatial light modulator 14 arecombined. In such embodiments, first spatial light modulator 14 maycomprise an array of light sources such as lasers that can be switchedon or turned off (or otherwise switched between light-emitting and darkstates).

Second spatial light modulator 20 comprises a plurality of controllableelements 22. Each controllable element 22 can be controlled to select aproportion of the light 25 that is incident on the element 22 from firstspatial light modulator 14 that is transmitted to a viewing area.

Second spatial light modulator 22 may be provided by any suitabletechnology, such as, for example:

a liquid crystal display (LCD) panel;

a liquid crystal on silicon LCOS chip;

a micro-mirror array;

magneto-optic devices;

light valves;

etc.

In some embodiments, second spatial light modulator 20 comprises opticalreflective or transmissive elements that can be switched between ON andOFF states. In such embodiments, second spatial light modulator 20 maybe controlled by a controller that sets its elements to be ON or OFF.

In some embodiments, first spatial light modulator 14 and second spatiallight modulator 20 each comprise a DMD or other two-dimensional array ofcontrollable micro-mirrors. Such embodiments have the advantage thatDMDs can be sourced relatively inexpensively and there is currently awide range of support for the design and manufacture of devices whichincorporate DMDs.

Transfer optics 26 carry light 25 from first spatial light modulator 14to second spatial light modulator 20. Light 25 is capable ofilluminating the entire active area of second light modulator 20 whenall elements 16 of first spatial light modulator 14 are ON. Light 25could spread past the edges of second spatial light modulator 20.

Transfer optics 26 blur light 25. Transfer optics 26 may becharacterized by a transfer function which at least approximates howlight 25 issuing from a point on first spatial light modulator 14 willbe spread over second spatial light modulator 20.

The pattern of light incident on second light modulator 20 can beestimated or determined from the configuration of first modulator 14(i.e. from which elements 16 are ON and which elements 16 are OFF) andthe transfer function.

It can be appreciated that, due to the blurring introduced by transferoptics 26, the light 25 incident on any element 22 of second spatiallight modulator 20 may arise from multiple elements 16 of first spatiallight modulator 14. The number of elements 16 of first spatial lightmodulator 14 that can contribute significant amounts of light to theillumination of an element 22 of second spatial light modulator 20depends primarily upon the width of the transfer function and the sizeof elements 16 of first spatial light modulator 14.

In some embodiments, first and second spatial light modulators 14 and 20have the same or similar numbers of controllable elements. In someembodiments, first spatial light modulator 14 has significantly fewercontrollable elements 16 than second spatial light modulator 20 hascontrollable elements 22. In some embodiments, first spatial lightmodulator 14 comprises an array of from about 140 to about 1600 elements16. Where the first and second spatial light modulators have differentspatial resolutions, in some embodiments the second spatial lightmodulator has the higher resolution and in some embodiments the firstspatial light modulator has the higher resolution.

In some embodiments, controllable elements 16 of first spatial lightmodulator 14 are arranged in a regular array. The array may berectangular and may comprise M rows and N columns of controllableelements 16. In some embodiments, controllable elements 22 of secondspatial light modulator 20 are arranged in a regular array. For example,the array may be rectangular and may comprise P rows and Q columns. Insome embodiments, second spatial light modulator 20 has a width andheight having a ratio of 16:9.

Some embodiments take advantage of the fact that a DMD or other spatiallight modulator having fewer elements in the same area may have a higherfill factor than a DMD or other spatial light modulator having moreelements in the same area. Thus, all other factors being equal, themaximum amount of light that a lower-resolution first spatial lightmodulator can pass on to a second spatial light modulator can be greaterthan the maximum amount of light that can be passed by ahigher-resolution spatial light modulator.

In some embodiments, the optical fill factor of a lower-resolution oneof the first and second spatial light modulators is at least 85%. Insome embodiments, the optical fill factors of both the first and secondspatial light modulators is at least 85%.

In some embodiments, first spatial light modulator 14 has a total numberof elements 16 that is at least a factor of two to four smaller than atotal number of elements 22 in second spatial light modulator 20. Theblur introduced by transfer optics 26 reduces or eliminates any‘blocking’ that could be caused by the low resolution of first spatiallight modulator 14.

Transfer optics 26 may comprise any suitable arrangement of lenses,mirrors, diffusers or the like which transfers light 25 originating fromfirst spatial light modulator 14 (primarily elements 16 that are intheir ON states) to second spatial light modulator 20. Some examples ofsuitable transfer optics 26 are:

-   -   a lens or system of lenses that projects an out-of-focus image        of first modulator 14 onto second modulator 20;    -   a lens or system of lenses in combination with a diffuser.

It is expedient to provide an optical system 26 for which the transferfunction is substantially the same for all elements of first modulator14. However, an optical system 26 that introduces both blur anddistortion could be used if the distortion can be characterized. It isalso expedient to provide an optical system 26 for which the transferfunction has circular symmetry. However, an optical system 26 that has amore complicated transfer function could be used as long as the transferfunction can be suitably characterized

As discussed below, some embodiments estimate the distribution of lightat second modulator 20 for different configurations of first modulator14. In such embodiments, it can be desirable to provide transfer optics26 characterized by a transfer function that blurs over a relativelysmall area as this reduces the computational requirements for estimatingthe resulting light field at second spatial light modulator 20. Inexample embodiments, the transfer function of transfer optics 26 may beapproximated to an acceptable degree of accuracy by a spatial low-passfilter or a smoothing operator characterized by a standard deviationlarger than the spacing between adjacent elements 16 of first spatiallight modulator 14.

Where display 10 is a projection-type display, a suitable projectionlens 28 focuses light from second spatial light modulator 20 onto ascreen 29 for viewing. Screen 29 may comprise a front-projection screenor a rear-projection screen.

In an example embodiment, first and second modulators 14 and 20 eachcomprise a DMD, and light source 12 comprises a laser light source.

FIG. 2 illustrates a method 40 for using a display like display 10 todisplay images. In block 42, image data 43 is provided. Image data 43defines an image to be displayed using display 10. For example, theimage data may specify a desired brightness as a function of positionfor each element of second spatial light modulator 20. Image data 43 maycomprise a frame of a video sequence, a still image, or the like. Theimage data may be represented in any suitable format. Some exampleformats in which image data may be presented are:

-   -   JPEG    -   JPEG-HDR    -   TIFF    -   GIF    -   OpenEXR    -   Artizen™ file format    -   Radiance™ file format    -   PNG (Portable Network Graphics),    -   bit-map (e.g. .BMP)    -   JPEG2000    -   MPEG    -   MPEG-HDR    -   DPX format (ANSI/SMPTE 268M-1994, SMPTE Standard for File Format        for Digital Moving-Picture Exchange (DPX), v 1.0, 18 Feb. 1994)    -   DCI digital cinema format    -   Cineon™ format    -   etc.        In some embodiments, the format is a high dynamic range (HDR)        format providing more than 24 bits per pixel.

In blocks 44 to 50, method 40 derives driving signals for the elements16 of first spatial light modulator 14. The driving signals can beapplied to set each element 16 to be ON or OFF in a pattern suitable forreproducing the image of image data 43. Block 44 determines grey scalebrightness levels that should be provided for each different area of anoutput image to be projected onto screen 29. Each of these areascorresponds to an area of first spatial light modulator 14. The areas offirst spatial light modulator 14 each encompass a plurality of elements16. Block 44 may, for example, comprise averaging together pixel valuesfor portions of the image defined by image data 43 that correspond toeach area of the output image.

Block 46 determines a pattern of ON and OFF states that can be appliedto the elements of first spatial light modulator 14 such that in each ofthe areas of first spatial light modulator 14, the proportion of ONelements 16 varies with the corresponding grey scale brightness leveldetermined in block 44. For example, for areas corresponding to brightportions of the image, the pattern may specify that all of the elements16 in the corresponding area of first spatial light modulator 14 shouldbe ON. For areas corresponding to dim portions of the image, the patternmay specify that most or all of the elements 16 in the correspondingarea of first spatial light modulator 14 should be OFF.

Where an area of the pattern corresponds to an intermediate brightnessthen an appropriate proportion of elements 16 in the area will be ON andthe remainder OFF. In this case it is desirable that the ON elements 16be reasonably evenly distributed over the area. For example, elements 16in the area may be distributed according to a suitable dithering patternthat has the desired ratio of ON to OFF elements 16.

A dithering pattern may be generated, for example, by:

-   -   generating a luminance map that indicates, for each pixel of the        image, how much luminance should be allowed to pass to a viewer;    -   boosting the luminance map to yield a boosted luminance map;    -   downsampling the boosted luminance map to a resolution matching        that of first spatial light modulator 14 to yield a downsampled        grey scale image; and    -   dithering the resulting downsampled grey-scale image to yield a        binary image.        Boosting the luminance map is desirable to ensure that there        will be sufficient light at each element of second spatial light        modulator 20 that the amount of light specified by image data 43        will be available to pass to a viewer.

Dithering may be performed in any suitable manner. Dithering softwareand hardware are commercially-available. In some embodiments, ditheringis performed for blocks of elements 16 on first spatial light modulator14. For example, dithering performed over a 16×16 block of elements 16can produce light outputs which vary in 256 steps from no output (apartfrom any leakage light) wherein all 256 elements in the block are OFF,to a maximum output level wherein all 256 elements in the block are ON.Dithering may comprise looking up predetermined dither patterns in atable or other suitable data structure or computing dither patternswhich provide the appropriate densities of ON elements 16 by applying asuitable dithering algorithm. The dithering algorithm may be implementedin software, hardware or a suitable combination thereof.

Some example dithering algorithms include:

-   -   Dividing an image into tiles, assigning a rounding bias to each        pixel position within a tile, adding the rounding bias to the        pixel value and then rounding the resulting value down. Each        pixel in the tile will then have a high value (e.g. “1” or ON)        or a low value (e.g. “0” or OFF).    -   Floyd-Steinberg dithering algorithms.    -   Average dithering (which may involve, for example, selecting a        threshold pixel value, which may be the average value of image        pixels and then quantizing pixels to low or high values (e.g. 0        or 1) based upon whether the values for the pixels are greater        than or less than the threshold), and using it as a global        threshold in deciding whether a pixel should be quantized to 0        or to 1. The case where the pixel value is equal to the        threshold may be handled in any suitable way. In an embodiment        all pixels whose values are above the threshold are quantized to        1 and all other pixels are quantized to a value of 0.    -   Random dithering.    -   Error-diffusion dithering.    -   Veryovka-Buchanan dithering algorithms.    -   Riemersma dithering.    -   etc.        Matlab™ and other computation and/or image processing software        packages include software which implements dithering algorithms        that may be used in embodiments of the invention.

FIG. 3A illustrates an image 55 divided into portions 57A, 58A, 59A and60A (each shaded differently to represent various grey scale brightnesslevels). Portion 57A is at a maximum (i.e. 100%) brightness level,portion 58A is at a 50% brightness level, portion 59A is at a 67%brightness level and portion 60A is at a minimum (i.e. 0%) brightnesslevel.

FIG. 3B illustrates one example of a dithering pattern 56 that may beapplied to first spatial light modulator 14 to yield a light fieldhaving portions with the brightness levels shown in FIG. 3A. Ditheringpattern 56 has areas 57B, 58B, 59B and 60B each having an 8×8 array ofpixels. Each pixel corresponds to one of the elements 16 of firstspatial light modulator 14 which may be set to ON (shown as an unshadedpixel) or OFF (shown as a shaded pixel).

The brightness level of a portion of image 55 (FIG. 3A) determines theproportion of pixels in an ON or OFF state in a corresponding area ofdithering pattern 56 (FIG. 3B). In area 57B, all of the pixels are setto ON to yield a maximum brightness level corresponding to portion 57A.In area 60B, all of the pixels are set to OFF to yield a minimumbrightness level corresponding to portion 60A. In area 58B, 50% of thepixels are set to ON to yield a brightness level corresponding toportion 58A. In area 59B, 67% of the pixels are set to ON to yield abrightness level corresponding to portion 59A.

In addition to the dithering pattern shown in FIG. 3B, various otherdithering patterns may be used to represent the image shown in FIG. 3A.For example, a different combination of pixels in area 58B may be set toON (the combination comprising 50% of the total pixels in area 58B) tomaintain the average brightness level of the area at 50%.

Block 48 predicts the amount of light 25 that will be incident on eachelement 22 of second light modulator 20 if the elements of firstmodulator 14 are set according to the pattern determined in block 46.This prediction may be made, for example, by applying a mathematicalfunction which approximates the transfer function of transfer optics 26to the pattern of light that would be produced at first spatial lightmodulator 14 by setting elements 16 according to the pattern determinedin block 46.

The light field estimation of block 48 may be performed at variouslevels of detail. In some embodiments, the light field estimation ofblock 48 may comprise upsampling, if necessary, a spatially-ditheredimage produced in block 46 to a resolution matching (or exceeding) thatof second spatial light modulator 20 and applying a smoothing functionsuch as a blur filter or low-pass filter to the result. In someembodiments the blur filter has a small kernel, such as a 3×3 or 5×5kernel. In some embodiments, the blur filter has a kernel not exceeding5×5. The smoothing function approximates the transfer function of optics26.

Block 50 determines the proportion of the incident light 25 that shouldbe allowed to pass each element 22 of second light modulator 20 to yielda desired image. Block 50 may comprise, for example, dividing abrightness value specified by image data 43 for an element 22 by thebrightness of the light 25 at that element 22 as estimated in block 48to yield a value indicating how much the element 22 should attenuate theincident light 25. The resulting set of values may be termed a‘correction mask’ because it corrects the blurry light field incident onsecond spatial light modulator 25 to yield the desired image. Block 50may optionally comprise subjecting the correction mask to a sharpeningoperation.

In block 52 the pattern derived in block 46 is applied to drive elements16 of first modulator 14 and in block 54 the values derived in block 50are applied to drive elements 22 of second modulator 20.

Blocks 52 and 54 occur at the same time. Where second modulator 20comprises a DMD or other modulator having binary elements 22 then block54 may comprise varying the proportion of time in which elements 22 arein their ON states. For example:

-   -   Elements 22 may be driven according to a suitable pulse-width        modulation (PWM) scheme.    -   Elements 22 may be driven according to a scheme by which they        are switched ON in each time period for a number of pulses which        depends on the corresponding value.    -   Elements 22 may be turned ON at the beginning of each time        period and then switched OFF after a portion of the time period        has elapsed that depends on the corresponding value.    -   etc.        Elements 16 of first modulator 14 may remain set in their ON or        OFF states as long as it is desired to display the image.

In an example embodiment, first spatial light modulator 14 substantiallycontinuously displays a spatial dither pattern during a frame,transmission optics 26 blur and project the light from first spatiallight modulator 14 onto second spatial light modulator 20 to yield ablurred grey scale image on second spatial light modulator 20 and theelements of second spatial light modulator are switched between their ONand OFF states during the frame to allow desired amounts of light toreach a viewer.

Method 40 may optionally be augmented, if desired, by controlling thebrightness of light source 12 in response to the brightest portions ofthe image to be displayed. Where the overall image is dark and does nothave any very bright parts, the intensity of light source 12 may bereduced. For images that include bright areas, light source 12 may beoperated at its full intensity.

FIG. 4 shows, for a line extending across an area of an example image,the following curves:

-   -   curve 60 representing the original image data;    -   curve 61 representing a spatially-dithered image (which would be        present at second spatial light modulator 20 if transfer optics        26 focused an image of first spatial light modulator 14 onto        second spatial light modulator 20);    -   curve 62 representing a luminance image at second spatial light        modulator 20 resulting from the spreading of light in the        spatially-dithered image by transfer optics 26;    -   curve 63 representing transmission levels for the elements of        second spatial light modulator 20;    -   curve 64 representing transmission levels for the elements of        second spatial light modulator 20 that have been sharpened; and    -   curve 65 (which coincides with curve 60 representing the        displayed image).

Displays which incorporate some or all of the concepts described hereincan be implemented in a wide variety of ways. Advantageously, firstspatial light modulator 14 does not need to be defect-free. Even ifoccasional individual elements 16 are stuck in their ON or OFFconfigurations, the blurring introduced by transfer optics 26 willensure that a few such individual-element defects do not have a largeadverse effect on the resulting image. If it is desired to explicitlyaccommodate defective elements then a number of options are possibleincluding:

-   -   maintaining a defect map indicating the state of any defective        elements 16 and taking these states into consideration when        performing light field estimation (e.g. in block 48); and,    -   maintaining a defect map indicating the state of any defective        elements 16 and arranging the pattern of ON elements 16 to take        these defective states into consideration. For example, if block        46 determines that in a particular area of first spatial light        modulator 14 half of elements 16 ought to be ON and the other        half of elements 16 ought to be OFF then block 46 may comprise        attempting to include in the pattern as being ON those defective        elements in the area that are stuck ON and including in the        pattern as being OFF those defective pixels in the area that are        stuck OFF.

FIG. 5 illustrates a display controller 70 according to an embodiment ofthe invention. Display controller 70 may be applied to drive the firstand second spatial light modulators of a display 10 for example. Displaycontroller 70 has an input 72 which receives image data 43 defining animage to be displayed. A codec 74 extracts a frame 75 of the image fromimage data 43. Frame 75 is made up of data that specifies a luminancevalue or equivalent for each position (x,y) in the frame. The datamaking up the frame is made available to a dithering engine 76 and acorrection mask generator 78.

Dithering engine 76 establishes a spatially-dithered pattern 77 at theresolution of first spatial light modulator 14 that corresponds to frame75. Pattern 77 is made available to a light field simulator 80 and afirst spatial light modulator driving circuit 82.

Light field simulator 80 estimates the light field at second spatiallight modulator 20 corresponding to pattern 77. The estimate 79 is madeavailable to correction mask generator 78. Correction mask generator 78computes desired transmission values for the elements 22 of secondspatial light modulator 20 to yield a correction mask 81 which is madeavailable to a second spatial light modulator driving circuit 84.Correction mask generator 78 generates correction mask 81 based at leastin part on frame data 75 and light field estimate 79.

First spatial light modulator driving circuit 82 is configured to setelements 16 of a first spatial light modulator 14 to be ON or OFF asspecified by pattern 77 and to hold those elements in the selected statefor the duration of a frame. Second spatial light modulator drivingcircuit 84 is configured to set the elements of second spatial lightmodulator 20 to have transmission values as specified by correction mask81. Where second spatial light modulator 20 comprises a DMD, secondspatial light modulator driving circuit 84 may rapidly switch elements22 between their ON and OFF states such that a ratio between the ON timeand OFF time for each element corresponds to a transmission value forthe element as specified in correction mask 81. Second spatial lightmodulator driving circuit 84 may comprise a PWM DMD driver circuit forexample. Circuits for driving DMDs are commercially available. Oneexample is the DMD Discovery™ chipset available from Texas Instruments.

A timing system 86 coordinates the operation of apparatus 70 such thatdriving signals for a frame are applied to first and second spatiallight modulators 14, 20 for the duration of the frame by drivingcircuits 82 and 84 respectively.

It is convenient but not mandatory that first spatial light modulator 14be driven throughout a frame. For example, it would make no differenceto the resulting image if first spatial light modulator 14 is not drivenduring any periods in which all elements of second spatial lightmodulator 20 are OFF.

The invention may be applied to color displays as well as to monochromedisplays. This may be achieved in various ways. One approach is todisplay different colors in a time-multiplexed manner. This may be doneby introducing different color filters into the optical path. Forexample, display 10 of FIG. 1 could be modified to include a colorwheel.

In some color displays, a plurality of color channels (for example, red,green and blue channels) are processed separately and the light from thedifferent color channels is combined at or upstream from a displayscreen to yield a color image. This invention may be practiced in thismanner. For example, FIG. 6 shows a color display 88 having red, greenand blue sections 90R, 90G and 90B (collectively sections 90)respectively. Each section 90 comprises a light source that produceslight of the corresponding color. The light sources may be separate ormay comprise suitable filters arranged to obtain light of the requiredcolor from a single white light source. In the illustrated embodiment,separate red green and blue light sources 12R, 12G and 12B are provided.

Each section 90 works in substantially the same manner as display 10described above except that the sections 90 are driven in response toimage data for the corresponding colors. The components of each section90 are identified with the same reference numbers as the components ofdisplay 10 with an R, G or B appended respectively.

FIG. 7 shows a color display 95 of an alternative design. Display 95 hasa light source 97 that illuminates a first spatial light modulator 99.First spatial light modulator 99 comprises an array of elements 100 thatare switchable between ON and OFF states. Light modulated by firstspatial light modulator 99 is directed to three second spatial lightmodulators 102A, 102B and 102C (collectively second spatial lightmodulators 102) by way of transfer optics 103 that comprises prisms 104,105A, 105B and filters 106 and 107.

Filters 106 and 107 cause light incident from first spatial lightmodulator 99 to be divided into three spectral components (for example,red, green and blue). Each spectral component is directed to andmodulated by one of second spatial light modulators 102A, 102B and 102C(collectively second spatial light modulators 102). Each spectralcomponent is characterized by a light field having a spatially-varyingintensity that is determined by the pattern of elements 100 set to ON infirst spatial light modulator 99. The light fields are blurred images offirst spatial light modulator 99 as delivered by transfer optics 103.

Light that has been modulated by second spatial light modulators 102passes out of prisms 104, 105A and 105B to a projection lens 110 andscreen 111. Screen 111 may comprise a front-projection screen or arear-projection screen.

During a frame, the elements of first spatial light modulator 99 ofdisplay 95 are set to display a pattern having a spatially-varyingdensity. The density may be based upon image data for an image to bedisplayed. In some embodiments, the density of elements 100 that are setto ON in an area of first spatial light modulator 99 may be determinedbased upon luminance values determined from the image data. Theparticular patterns of elements 100 that are set to ON to achieve thedesired densities may be determined by applying a suitablespatial-dithering algorithm or spatial-dithering engine, for example tothe image data. The methods and apparatus described above forcontrolling first spatial light modulator 14 of display 10 may beapplied with suitable modification for controlling first spatial lightmodulator 99 of display 95.

Second spatial light modulators 102A, 102B and 102C may be controlled insubstantially the same manner as second light modulator 20 of display 10with the exception that transmission values for the elements of secondspatial light modulators 102A, 102B and 102C are determined based uponinformation in the image data for the corresponding spectral components.

FIG. 8 is a block diagram of a control system 112 for a display likedisplay 95. Image data 113 is received at input 114. Luminanceinformation 115 for a frame is extracted and provided to a patterngenerator 116. Pattern generator 116 outputs a pattern 117 having aspatially-varying density based on the luminance data. Pattern 117 isapplied to a driving circuit 118 for first spatial light modulator 99.

Pattern 117 is also provided to light field estimator 119 which outputsan estimated light field 120. If the optical transmissioncharacteristics of the optical paths between first spatial lightmodulator 99 and second spatial light modulators 102A, 102B and 102C aredifferent, light field estimator 119 may generate a separate estimatedlight field (120A, 120B and 120C) corresponding to pattern 117 for eachsecond spatial light modulator 102A, 102B and 102C.

Color information for a frame comprising first, second and thirdspectral component color information (121A, 121B and 121C respectively)and the corresponding estimated light field 120 (or 120A, 120B and 120C)are provided to correction mask generators 122A, 122B and 122C.Correction mask generators 122A, 122B and 122C generate correction masks123A, 123B and 123C that are provided to driving circuits 125A, 125B and125C (collectively driving circuits 125) which can drive second spatiallight modulators 102A, 102B and 102C respectively.

A display like display 95 may be operated by a method similar to method40 of FIG. 2.

In some embodiments, second spatial light modulators 102 are DMDs. Insome embodiments driving circuits 125 are PWM driving circuits.

FIG. 9 shows a color display 130 of another alternative design having anumber of first spatial light modulators 132A, 132B and 132C(collectively first spatial light modulators 132) and a second spatiallight modulator 134 that further modulates light from first spatiallight modulators 132 after the light has been combined to form a colorimage. Display 130 has an optics subsystem 137 which receives lightincident from light source 135. Optics subsystem 137 has a plurality ofprisms 138 and filters 140 for dividing the light from light source 135into three spectral components (for example, red, green and blue) anddirecting each spectral component to a corresponding one of firstspatial light modulators 132A, 132B and 132C. Each of first spatiallight modulators 132A, 132B and 132C has an array of elements that areswitchable between ON and OFF states for modulating light passed to themodulator. The light which is modulated by first spatial lightmodulators 132A, 132B and 132C is then combined into a color image andcarried by transfer optics to second spatial light modulator 134. Thetransfer optics blur the light. Light that has been modulated by secondspatial light modulator 134 passes to a projection lens 145 and screen146.

First and second light modulators are driven in a manner that causes theimage projected onto screen 146 to reproduce a desired image specifiedby image data.

To simplify the explanation of the embodiments of the inventiondescribed above, various elements that are commonly found in projectorsand other DMD-based devices and that may also be present in displaysaccording to this invention are not specifically described. Suchelements are known to those of skill in the art of designingprojection-type displays. Some examples are: power supplies, coldmirrors to direct infrared radiation out of the optical path, integratorrods to collect light for illuminating a DMD, bending optics, housings,user controls, etc.

As will be understood from the foregoing description, the are a numberof cases in which the designs and methods described herein may beusefully applied. Some examples include:

-   -   A monochrome display in which a first binary light modulator        modulates light from a light source and that light is further        modulated by a second binary light modulator.    -   A color display in which a first binary light modulator        modulates light from a light source, that light is further        modulated by a second binary light modulator and the color of        light form the light source is switched between sub-frames. For        example, the light source may comprise separate red, green and        blue light sources that each illuminate the first modulator only        during a corresponding sub-frame or light from a white light        source is directed to pass through a color wheel before it        illuminates the first light modulator or the like. This color        implementation has the advantage of simplicity but requires that        the modulators can be updated at high speed to permit red-        green- and blue-sub-frames to be displayed sequentially at a        rate fast enough to provide a satisfactory viewing experience.    -   Separate first and second binary light modulators may be        provided for each of a plurality of color channels. For example,        separate red- green- and blue-color channels may each have first        and second binary modulators arranged and operated as described        herein. The images from the color channels may be optically        superposed to achieve a color image. Light for each of the color        channels may be provided from separate light sources or by        splitting white light into the required number of color bands.        Such embodiments may advantageously provide high brightness but        can be more expensive to make (as they require 6 modulators and        associated control circuitry and optics to control three color        channels).    -   A first binary light modulator may modulate light from a light        source that emits light having multiple color components. The        light may then be split into separate color components and each        of the color components (e.g. red- green- and blue components)        are directed to a corresponding second binary light modulator.        The light modulated by the second binary light modulators is        combined to provide a color image. The first binary light        modulator has a spatial resolution that is significantly lower        than that of the second binary light modulators in some        embodiments.    -   Separate first binary light modulators may be provided for each        of a plurality of color channels. For example, separate red-        green- and blue-color channels may each have a first binary        modulator. Light for each color channel may be provided by a        separate light source or by splitting light from a white or        other multi-component light source into a plurality of color        bands. Light modulated by the first binary light modulators may        be optically combined and the combined light illuminates a        second binary light modulator which modulates the combined light        to provide a color image. The first and second binary modulators        are arranged and operated as described herein. In some        embodiments, the second spatial light modulator has a spatial        resolution significantly less than that of the first spatial        light modulators.    -   In either of the two immediately-preceding embodiments, a single        binary light modulator, which may have a relatively low spatial        resolution, acts on the combined image (luminance modulation        only) while separate modulators provide modulation for each        color channel. The color modulation provided by the separate        modulators may have a relatively high spatial resolution.    -   In the embodiments described above in this paragraph, where the        first and second binary modulators have different spatial        resolutions, the lower-resolution one(s) of the binary        modulators may, for example, have resolutions (e.g. numbers of        controllable elements) that are a factor of 64 or more or, in        some cases, 1024 or more lower than the spatial resolution of        the higher-resolution modulator.    -   In some further embodiments, separate first binary light        modulators are provided for each of a plurality of color        channels. For example, separate red- green- and blue-color        channels may each have a first binary modulator. Light for each        color channel may be provided by a separate light source or by        splitting light from a white or other multi-component light        source into a plurality of color bands. Light modulated by the        first binary light modulators may be optically combined and the        combined light illuminates a second binary light modulator which        modulates the combined light to provide a color image. The first        and second binary modulators are arranged and operated as        described herein. In this embodiment, the first binary light        modulators which each modulate light in one color channel may        have spatial resolutions that are smaller than that of the        second spatial light modulator. Preferably, the spatial        resolutions of the first color light modulators are at most        about 2 to 6 times lower than the spatial resolution of the        second spatial light modulator. The human visual system is more        sensitive to local luminance changes than it is to local (high        spatial frequency) color changes.    -   A first binary light modulator may modulate light from a light        source. Light from the first binary light modulator may be        blurred and passed to a second light modulator comprising an LCD        panel or other modulator capable of controlling transmission of        light continuously or in multiple steps over a reasonably wide        range.    -   Light modulation methods as described herein may be performed on        separate channels for left and right images in 3D digital cinema        systems. The left and right images may be differently polarized,        have different spectral characteristics or be displayed in a        time interlaced manner. Suitable polarizing or spectral filters        may be provided in each channel.

Where a component (e.g. a software module, processor, assembly, device,circuit, etc.) is referred to above, unless otherwise indicated,reference to that component (including a reference to a “means”) shouldbe interpreted as including as equivalents of that component anycomponent which performs the function of the described component (i.e.,that is functionally equivalent), including components which are notstructurally equivalent to the disclosed structure which performs thefunction in the illustrated exemplary embodiments of the invention.

Where a controller for a display as described herein is implemented insoftware, the controller may comprise a data processor and softwareinstructions stored on a tangible medium accessible to the dataprocessor. The data processor may generate first signals for the controlof one or more first spatial light modulators and second signals for thecontrol of one or more second spatial light modulators by executing thesoftware instructions to process image data to yield the first andsecond signals. In alternative embodiments, fixed or configurablehardware such as logic circuits or a field-programmable gate array(FPGA) are provided to perform some or all steps in processing the imagedata to yield the first and second control signals.

As will be apparent to those skilled in the art in the light of theforegoing disclosure, many alterations and modifications are possible inthe practice of this invention without departing from the spirit orscope thereof. For example:

-   -   For any desired spatial density of ON elements in the first        spatial light modulator (except for all elements ON or all        elements OFF) there are a variety of patterns that can be        applied to an area of the first spatial light modulator. It is        possible to switch the elements of the first spatial light        modulator to change the pattern of ON elements without varying        the spatial density of ON elements during a frame without        adversely impacting the displayed image.        Accordingly, the scope of the invention is to be construed in        accordance with the substance defined by the following claims.

What is claimed is:
 1. A display comprising: a light source capable ofdirecting light onto a first spatial light modulator, the first spatiallight modulator comprising a plurality of first reflective elementsswitchable between ON and OFF states, transfer optics disposed to directlight modulated by the first spatial light modulator onto a secondspatial light modulator, and a driver configured to generate first andsecond control signals for the first and second spatial light modulatorsrespectively based on image data, wherein: the driver is configured togenerate a pattern based upon the image data, the pattern having aspatially-varying density and the driver is configured to generate thefirst control signal so as to set elements of the first spatial lightmodulator according to the pattern and to hold the elements of the firstspatial light modulator set according to the pattern while applying thesecond control signals to the second spatial light modulator; and, thetransfer optics are characterized by a transfer function that blurslight originating from the first spatial light modulator at the secondspatial light modulator.
 2. A display according to claim 1 wherein thesecond spatial light modulator comprises a plurality of elementsswitchable between ON and OFF states and the driver is configured toswitch the elements of the second spatial light modulator between theirON and OFF states multiple times while the elements of the first spatiallight modulator remain set according to the pattern.
 3. A displayaccording to claim 2 wherein the driver is configured to switch theelements of the second spatial light modulator between their ON and OFFstates according to a binary pulse-width modulation scheme.
 4. A displayaccording to claim 1 comprising a dithering engine wherein the patterncomprises an output of the dithering engine.
 5. A display according toclaim 1 wherein the first and second spatial light modulators eachcomprise a digital mirror device.
 6. A display according to claim 1wherein the light source comprises a laser light source.
 7. A displaycomprising: a light source capable of directing light onto a firstspatial light modulator, the first spatial light modulator comprising aplurality of first reflective elements switchable between ON and OFFstates, transfer optics disposed to direct light modulated by the firstspatial light modulator onto a second spatial light modulator, and adriver configured to generate first and second control signals for thefirst and second spatial light modulators respectively based on imagedata, wherein: the driver is configured to generate a pattern based uponthe image data, the pattern having a spatially-varying density and thedriver is configured to generate the first control signal so as to setelements of the first spatial light modulator according to the patternand to hold the elements of the first spatial light modulator setaccording to the pattern while applying the second control signals tothe second spatial light modulator; the transfer optics arecharacterized by a transfer function that blurs light originating fromthe first spatial light modulator at the second spatial light modulator;and the driver is configured to estimate a light field at the secondspatial light modulator corresponding to the pattern and to base thesecond control signals on the estimated light field.
 8. A displayaccording to claim 7 wherein the second spatial light modulatorcomprises a plurality of elements switchable between ON and OFF statesand the driver is configured to switch the elements of the secondspatial light modulator between their ON and OFF states multiple timeswhile the elements of the first spatial light modulator remain setaccording to the pattern.
 9. A display comprising: means for generatinglight; first means for binary modulation of the light, the first meanscomprising a plurality of first reflective elements switchable betweenON and OFF states; means for blurring light modulated by the first meansand directing the blurred light onto a binary spatial light modulator;means for generating first control signals for the first means based onthe image data, the means for generating first control signalscomprising means for generating a pattern based upon the image data, thepattern having a spatially-varying density; and means for generatingsecond control signals for the binary spatial light modulator based onthe image data.
 10. A display according to claim 9, comprising means formaintaining the elements of the first means in states according to thepattern while operating the binary spatial light modulator to modulatethe blurred light according to the second control signals.
 11. A displayaccording to claim 10 comprising means for estimating a light field ofthe blurred light at the binary spatial light modulator wherein themeans for generating second control signals is responsive to both theimage data and the estimated light field.
 12. A method for displaying animage, the method comprising: setting reflective elements of a firstbinary spatial light modulator according to a binary pattern based onthe image, the pattern having a spatially-varying density; blurring andtransferring to a second spatial light modulator an image of the firstbinary light modulator to yield a light field at the second spatiallight modulator; modulating the light field with the second spatiallight modulator to yield a reconstruction of the image while holding thereflective elements of the first binary spatial light modulator setaccording to the binary pattern.
 13. A method according to claim 12wherein modulating the light field with the second spatial lightmodulator comprises performing temporal dithering of the light field byswitching elements of the second spatial light modulator between ON andOFF states.
 14. A method comprising: setting reflective elements of afirst binary spatial light modulator according to a binary pattern basedon the image, the pattern having a spatially-varying density; blurringand transferring to a second spatial light modulator an image of thefirst binary light modulator to yield a light field at the secondspatial light modulator; modulating the light field with the secondspatial light modulator to yield a reconstruction of the image whileholding the reflective elements of the first binary spatial lightmodulator set according to the binary pattern; and computing an estimateof the light field corresponding to the pattern and controlling thesecond spatial light modulator according to the image data and theestimate of the light field.
 15. A method according to claim 14 whereinmodulating the light field with the second spatial light modulatorcomprises performing temporal dithering of the light field by switchingelements of the second spatial light modulator between ON and OFFstates.
 16. A display comprising: a first spatial light modulatorcomprising a plurality of first reflective elements switchable betweenON states, wherein the elements emit light, and OFF states wherein theelements are dark, transfer optics disposed to direct light from thefirst spatial light modulator onto a second spatial light modulator, anda driver configured to generate first and second control signals for thefirst and second spatial light modulators respectively based on imagedata, wherein: the driver is configured to generate a pattern based uponthe image data, the pattern having a spatially-varying density; thedriver is configured to generate the first control signal so as to setelements of the first spatial light modulator according to the patternfor the duration of an image frame; the transfer optics arecharacterized by a transfer function that blurs light originating fromthe first spatial light modulator at the second spatial light modulator;and, the second spatial light modulator comprises a plurality ofelements switchable between ON and OFF states and the driver isconfigured to switch the elements of the second spatial light modulatorbetween their ON and OFF states multiple times during the image frame.17. A display according to claim 16 wherein the display comprises alight source illuminating the first spatial light modulator and theelements of the first spatial light modulator pass light from the lightsource to the second spatial light modulator when in the ON states. 18.A display according to claim 17 wherein the first spatial lightmodulator comprises a digital mirror device.
 19. A display according toclaim 16 wherein the driver is configured to switch the elements of thesecond spatial light modulator between their ON and OFF states accordingto a binary pulse-width modulation scheme.
 20. A display according toclaim 16 comprising a dithering engine wherein the first control signalcomprises pattern output by the dithering engine.
 21. A displayaccording to claim 16 wherein the spatially-varying density of thepattern has a spatial resolution smaller than a spatial resolution ofthe second spatial light modulator.
 22. A display comprising: a firstspatial light modulator comprising a plurality of first reflectiveelements switchable between ON states, wherein the elements emit light,and OFF states wherein the elements are dark, transfer optics disposedto direct light from the first spatial light modulator onto a secondspatial light modulator, and a driver configured to generate first andsecond control signals for the first and second spatial light modulatorsrespectively based on image data, wherein: the driver is configured togenerate a pattern based upon the image data, the pattern having aspatially-varying density; the driver is configured to generate thefirst control signal so as to set elements of the first spatial lightmodulator according to the pattern for the duration of an image frame;the transfer optics are characterized by a transfer function that blurslight originating from the first spatial light modulator at the secondspatial light modulator; and, the second spatial light modulatorcomprises a plurality of elements switchable between ON and OFF statesand the driver is configured to switch the elements of the secondspatial light modulator between their ON and OFF states multiple timesduring the image frame; wherein the driver is configured to estimate alight field at the second spatial light modulator corresponding to thepattern and to base the second control signals on the estimated lightfield.